Subjective Evaluation of Mobile Speech Input: Experiences from Two Case Studies

Thorough understanding of subjective and objective measurements of speech-based interaction, especially of its user experience, is vital for practical application development. We present findings from two case studies where multimodal applications containing speech input were evaluated using a subjective evaluation methodology. Responses were investigated using both correlation analysis and exploratory factor analysis. The results from the correlation analysis suggest that previous experience with multimodal technologies does not overall significantly influence expectations or perceptions of actual use. Results from the factor analysis process aided us in characterizing the nature of user experience and serve as a starting point for further analyses. Finally, our results suggest that there is little correlation between subjective and objective metrics.